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Abstract 

Discussions of the functions of institutional research are common. However, 
discussions concerning the structures associated with disciplined inquiry, the 
research part of institutional research, have been rare. This paper compares the 
process and structure of institutional research with three ways in which science is 
conceptualized. These comparisons result in the recognition that although 
institutional research has much in common with science, it is distinct from science. 
This paper begins the process of identifying the structures and assumptions that 
form the basis of institutional research. 
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Institutional research is a term that is used in reference to a wide variety of 
activities ranging from data processing to disciplined inquiry. Several people have 
attempted to define institutional research — Maasen (1986) in terms of the main data 
handling activities; Peterson (1985) in terms of its linkages between institutions of 
higher education and the functions of higher education; Saupe (1981) in terms of its 
use in planning, policy analysis, and decision making. Definitions have failed to 
characterize important aspects of institutional research. "This...has become 
something of an embarrassment to those of us who have spent a number of years 
calling ourselves institutional researchers" (Muffo and McLaughlin, 1987, p. iv). 
The attempts at definition fail to address the process and structure of the research 
part of institutional research. 

An "ideal type" of disciplined inquiry has yet to be identified to guide the 
practitioner in doing institutional research. "Ideal type" is a term used by Max 
Weber (1949) to describe an abstract Utopian construct that characterizes a set of 
relationships. "It is not a description of reality but it aims to give unambiguous 
means of expression to such a description" (Weber, 1949, p. 90). A logical 
structure or model would be an ideal type. Like a map, it communicates structural 
essentials, thus reducing ambiguity. Having an ideal type allows one to better 
know how to go about doing what needs to be done. 

When one speaks of disciplined inquiry or research, the natural assumption 
is that one is talking about science. Science provides the ideal type that comes to 
mind when such things are discussed. For example, von Vught (1990) compared 
institutional research to science, and argued that institutional research faces 
legitimacy problems with the scientists that make up the academic community. 
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(Inherent in this argument is that institutional research should be compared to 
science, and in fact should be science.) 

Upon reflection, we find at least three ideal types associated with science. 
The first is the structure of the process of modern science, usually referred to as 
scientific method. The second is the logical structure of science. This ideal type 
deals with scientific ways of thinking. The third ideal type is related to the social 
structure of science and the social elements that contribute to the growth and 
development of an understanding of a set of related phenomena. 

This paper outlines the ideal types mentioned above and discusses 
institutional research's characteristics in relation to these structures of science. The 
first section of the paper examines scientific method as a process of disciplined 
inquiry, then compares institutional research to that process. The second section 
compares the logical structure of institutional research with the logical structure of 
scientific inquiry. The third section utilizes Thomas Kuhn's (1962) notion of 
scientific paradigms examine the community or institutional researchers. It is hoped 
that this discussion will lead to identification of ideal types in institutional research, 
dispelling misconceptions about the ' research" in institutional research and resulting 
in improvements in the quality of work done in the area. 

SCIENTIFIC METHOD 

Like institutional research, science as a form has itself defied simple 
definition. Philosophers of science have for centuries refined and reshaped the 
concept of science, and have argued about its features as a disciplined method of 
inquiry. At different times science has meant different things. It has undergone 
paradigm shifts — dramatic alterations of focus and perspective. With each shift, it 
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has incorporated earlier versions of science to some degree, but has taken a new 
approach in order to correct for weaknesses perceived in the previous paradigm. 
With each new approach, either the previous paradigms have been abandoned or 
they have been accepted as tools to be used in the quest for knowledge. In spite of 
the dynamic natures of science and institutional research, or perhaps because of 
them, science and institutional research have a great deal in common. 

"Scientific Method" is a logical process with which institutional research 
shares several features. Scientific method consists of the following: observations, 
facts, hypotheses, experiments, theories and laws. The logical structure begins not 
with theories, but rather first with observations and facts (Casti, 1989). 
Observations are more than just catalogued events, they often include groupings of 
data. Observations are made that single pieces of data group in a way that suggests 
to the observer that the grouping is meaningful. Conjecture about the explanations 
for the patterns observed leads to hypotheses. Experiments are conducted to test 
the hypotheses. If a hypothesis is supported by the evidence, the identification of 
laws and theory building follow. 

Institutional research follows a similar logical process. Observations and 
facts serve as the foundation for disciplined inquiry. Data and facts about the 
institution and about higher education are examined for patterns. It is the observers 
(which may include administrators and faculty, as well as the institutional 
researcher) who decide which information is relevant to the question at hand. The 
way in which the question is posed influences the understanding of what is 
relevant. The environmental factors that influence decision making also determine 
what is relevant. Pattern identification, the result of repeated observations in a 



o 

ERIC 



7 



Science & Institutional Research 

6 



context created by fact, is an important part of institutional research. (Sometimes 
the institutional research function ends here and others interpret the information as 
they see fit, but not always.) As with science, conjecture about the patterns of data 
can lead to hypotheses. "Experiments" are sometimes conducted to test the 
hypotheses. However, that is a point at which science and institutional research 
diverge, at least in the eyes of the traditional scientist. 

Laws and theories have not been identified for c. discipline of institutional 
research. This may be due to the fact that few attempts have been made to look for 
common principles that underlie inquiries into the same types of phenomena. 
However, some argue that there are no true laws associated with human social 
behavior. (See Nagel, 1961, pp. 447-546). A law is "a statement of a relation or 
sequence of phenomena invariable under the same conditions." "Theory" is defined 
as "a group of general propositions used as principles of explanation for a class of 
phenomena." (Random House College Dictionary). Institutional research does not 
at this point have laws and a body of theory that are specific to a particular 
discipline. However, institutional research does share some assumptions with the 
social sciences. They are as follows: 

1 . Human behavior can to some extent be observed, measured and 
predicted. 

2 . There are environmental factors that influence behavior. 

3 . Some environmental factors that influence behavior can be 
manipulated. 

4 . Manipulation of environmental factors, including social factors, can 
lead to changes in human behavior. 
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5 . Certain environmental and social factors have more influence on 
human behavior than others. 

6 . Social factors associated with certain characteristics of individuals 
may influence the ways in which the individuals behave. 

7 . Many of an individual's characteristics associated with behavior 
cannot be changed. 

As in the social sciences, many of the relationships identified in institutional 
research are probabilistic in nature (See Nagel, 1961, pp. 22-23). "Though the 
premises are logically insufficient to secure the truth of the explicandum, they are 
said to make the latter 'probable'" (Nagel, 1961, p.22). 

One reason laws and theories of institutional research have not been 
identified is that they have been looked for in the wrong places. Laws and theories 
are usually in the domain of the empirical. However, the laws and theories of 
institutional research lie in the realm of the normative. Some of social science is 
involved in the process of envisioning an ideal, what could be, given the right 
circumstances. An example is Democratic Theory, the Utopian ideal of government 
that consists of the following three elements: popular sovereignty, majority rule, 
and equality. Political scientists make observations about how and to what extent 
governments approximate the Utopian ideal of democracy. Institutional researchers 
often operate under the assumption that there is an ideal condition, predetermined 
by the mission, goals, and policies of their college or university, or by the 
immediate needs of the administration. They attempt to find the appropriate 
information that will contribute to understanding and working toward that ideal. 
The policies, mission, and goals of the particular institution serve as the body of 
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theory for that institution (not for a discipline of institutional research). We will call 
this the theory of the institution. The functions of institutional research are geared 
to this level. The theory of the institution is concerned with the understanding of 
the institution's workings and identity. 

Strategic planning and management are the processes through which new 
policies (potential contributions to the theory of the institution) are tested for their 
ability to contribute to reaching the ideal as stated in the institution's mission and 
goals. They make up the normative theory of institutional research. We will call 
this the theory of institutions of higher education. This body of theory is concerned 
with the processes used to arrive at findings, rather than findings or a generalization 
of findings. 

Laws and bodies of theory have not been identified in institutional research 
partly because of the type.; of questions institutional research attempts to answer. 
Science in general is intended to respond to the question "why?" However, 
institutional research in its capacity of decision support rarely approaches an issue 
from an explanatory perspective, but rather responds to the question "which?" 
When an issue arises concerning a policy decision, the decision makers want to 
know which choice is the best one when all of the facts available are taken into 
account. Included within the "which" questions are often smaller questions: 
"who?", "when?", "where?", "how much?", "did it work?" and "does it matter?" 
Science, on the other hand, attempts to discover a cohesive set of truths about the 
nature of reality. In science there is no intent to solve practical problems, only 
attempts to contribute to knowledge. 

A useful analogy may be to imagine that science is an investigation into how 
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a tool functions or why it works. For example, mathematicians, in their research, 
may focus on how and why a particular type of statistical operation works. 
Institutional research, in contrast, assumes that the tool works and applies the tool 
for its intended purpose to produce something that will be used within a specific 
real world context. Science creates knowledge, while institutional research goes 
one step further by using the knowledge created through science to meet practical 
ends. Institutional research is science applied. 

Earlier, we mentioned that institutional research does not have laws and 
theories in the same sense that we think of science as having laws and theories. 
This is true, but there are some things that are invariable and act as principles for 
explaining sets of phenomena in institutional research. Underlying the mission and 
goals of any institution are two things that do not change in concept, intent toward 
student success and the maintenance of organizational viability. These figure into 
the decisions made in an institution of higher education. How student success is 
defined or measured may depend upon the culture of the particular institution. The 
strategies used in attempts to maintain the viability of the organization are likely to 
differ from institution to institution and from one time to another. The constraints 
on the institution may be location or constituency specific. Nevertheless, all 
institutions of higher learning share the assumptions that encouraging student 
success and maintaining organizational viability are inviolable principles. 

In the place of theories in institutional research are the assumptions of what 
is appropriate, important, and beneficial to the unending pursuit of student success 
and organizational viability. Sometimes, the ways in which these are characterized 
or envisioned change with what is in vogue in higher education, or with the socially 
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or politically popular issue of the day. Planning to cope with faculty shortages, 
providing incentives for increasing cultural diversity, and buffering against budget 
turbulence are examples of such current issues in institutional research. 

Even though these driving principles of institutional research do not at this 
point make up a cohesive body of knowledge as do laws and theories in science, 
institutional research is not far removed from science. Any one of the topics of 
study of institutional research can be broken down into topics of narrow scope 
suitable for scientific study, and investigated using the Jaws and theories of one or 
more of the social sciences for the purpose of adding to understanding of how the 
world works. Some argue that they should be in order to improve the validity of 
institutional research. 

THE STRUCTURE OF SCIENCE 

The logical structure of science, as we know it today, is based on deduction 
or induction, and experimentation, modelling, and refutation. Each of the building 
blocks of the form of science was the result of an attempt to correct for flaws in 
previously used methods of explaining reality. 

Deduction was the "science" of Aristotle's day. In deduction, premises are 
postulated in such a way as to lead to a conclusion that is supported by the 
premises. The truth of the premises is simply assumed. In institutional research, 
the assumptions more than the facts are responsible for many conclusions. The 
truth of the premises that information kept on databases about students, faculty, 
facilities, and budgets provide meaningful and appropriate answers is assumed. 
That the mission and goals of the institution are appropriate to achieving student 
success and ensuring organizational viability is usually assumed. Most important, 
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it is assumed tnat applying conventions of science to decision making leads to better 
decisions than reading tea leaves, or flipping coins. 

Deduction moves from generalities to conclusions about a specific. The 
most famous example of a deductive argument follows: 

All men are mortal. 

Socrates is a man. 

Therefore, Socrates is mortal. 
Although deduction is less used in institutional research than induction for arriving 
at conclusions about the conditions with which the institution must cope, it is not 
uncommon. An example used in institutional research might look something like 
the following: 

All students must be able to read and write well enough to pass a 
minimum skills test in order to succeed in school 
John Doe cannot pass the minimum skills reading and writing test. 
Therefore, John Doe cannot succeed in school. 
Such assumptions may be a part of an assessment procedure used for 
admissions purposes or for evaluating the effectiveness of a program. The test's 
validity (of various types) is assumed. The relationship between test performance 
and success in school is assumed. The truth of the premises is based upon a set of 
decisions made in an artificial environment instead of being based upon assumed 
truths about the nature of reality (but philosophical arguments about differences 
between created reality and perceived reality are beyond the scope of this paper). 

Induction, a concept advocated by Francis Bacon for understanding the 
reality of the world through the facts rather than through preconceived ideas about 



13 



Science & Institutional Research 

12 



the nature of reality, relies on repeated observations of facts. These observations 
are then used as the basis for predictions. The following is an example of 
induction: I have observed that for 30 days the sun has set in the west. Therefore, 
based on these observations, I predict that the sun will set in the west, tomorrow. 
In institutional research, induction is essential to informing decision making. 
Trends are frequently used for projecting such things as enrollments, financial 
needs, market share, attrition, and other factors that colleges and universities must 
adjust or adjust to for the sake of maintahing viability. 

Galileo brought a new perspective to science when he introduced the 
controlled experiment. Controlled experiments improved upon induction and 
deduction by narrowing the number of possibilities in establishing causation. With 
deduction, the relationship was built into the argument as an assumption. With 
induction it was assumed that a series of like events had the same cause, although 
causation might never have been addressed in the process of inquiry. The modem 
conception of science began with the introduction of study of causal relationships. 

Although institutional research makes use of experimentation to some 
degree, it is like the majority of the social sciences in that it is dependent upon using 
quasi-experimental and non-experimental designs for attempting to understand the 
relationships between events. More often than not the best that can be achieved in 
the social sciences is the establishment of a probabilistic relationship between a 
group and one who shares certain characteristics of that group. (Nagel, 1961). 

Institutional research suffers from a variety of methodological problems that 
go beyond even those of the social sciences. One major difficulty with which 
institutional research must deal is that the entire population is often the sample upon 



14 



Science & Institutional Research 

13 



which inferences are based. Another is that obtaining sufficient information about 
the possible influences upon events is often too costly or too time consuming to be 
of benefit to the institution. The greatest problem is the reactive nature of 
institutional research. It is often the school's administration that decides which 
variables would be most useful to understanding the relationships between 
observed events, possibly resulting in "looking where the light is best," rather than 
looking where the answers are most likely to lie. The formulation of the question 
determines the kinds of relationships examined. 

Modelling added to science by formalizing a method for representing the 
relationships between observations. Modelling reduces problems of interpretation 
of relationships. It provides a "language" upon which all can agree, thereby 
making it possible for others to converse about relationships between observations 
in a way that is free from prejudice. Formal mathematical models and statistics and 
diagrams are the languages used to describe the nature of relationships. 

Modelling often provides the most convincing argument for choosing one 
policy alternative over another in institutional research. Modelling, like scientific 
method, reduces the information necessary for understanding to a manageable 
amount. Although the decision makers in the institution may have little 
understanding of statistics, they may understand a few key concepts, such as 
significance, correlation, and reduction of error. They may respond to charts and 
diagrams better than numbers. Once a model is built, it may be used repeatedly for 
evaluating and for projecting. 

It is the use of statistical techniques that leads some who participate in 
institutional research to believe that they are practicing science. Knowledge of and 
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skill in the use of sophisticated techniques are erroneously equated with an 
understanding of how science works. However, statistics and formal mathematical 
modelling are simply tools. They can be used hi the pursuit of scientific knowledge 
and in the process of arriving at practical solutions, but they are not the same as 
science. That is not to say that mathematics and statistics cannot be the topic of 
science. Mathematicians do attempt to explain why and how mathematics and 
statistics can and do work. Nonetheless, in the structure of science, and in 
institutional research, such modelling techniques are the products of science applied 
as tools, and not science. 

Refutation is the final major building block in the structure of science as it is 
practiced today. A hypothesis must be falsifiable or testable in order to make a 
contribution to knowledge. Popper was the philosopher who brought this concept 
to the fore. This view became popular as a result of the problems that arose in 
ruling out alternative explanations for phenomena. Popper realized that it only takes 
one piece of negative evidence to refute a hypothesis, while no amount of evidence 
will prove a hypothesis. He argued, "The criterion of the scientific status of a 
theory is its falsifiability or refutability, or testability" (Casti, 1989, p. 33). 

Testability is central to the function of institutional research. However, 
testability has a slightly different meaning for institutional research. The 
"hypotheses and theories" tested are the ones associated with the particular 
institution. Institutional research exists to inform decision making. If there were 
no alternatives, there would be no reason to make a decision. If there were no way 
to demonstrate th>; superiority of one alternative over another, flipping a coin would 
be adequate decision support. 
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Institutional research and science are not one and the same. That is not 
necessarily to be perceived as a disadvantage f ~ r either one. Pure or basic science 
is ill suited to the needs for which institutional research was created. The goal of 
institutional research is to reduce uncertainty in practical decision making. The 
rigors of science, along with its goal of contributing to knowledge, are much too 
limited for this purpose. Institutional research deals with more than a single 
question at a time; instead it deals with an "ecology " (Cleveland, 1988, p. 584) in 
which a balance must be struck among conflicting factors and factions. Finances, 
quality of instruction, market share, cultural diversity, student needs, and the 
influences on a.id of other social and environmental entities are often aspects of an 
issue that must be addicssed in order to inform a single decision. 

Institutional research is more an art than a science. It requires 
understanding of models, techniques, theories and laws from other disciplines to 
inform decision making. However, sorting out what is important to the 
institution's viability requires more. Insights into the institutions character, culture, 
history and environment are necessary to understanding the state of the institution's 
ecology and the nature of the question to be answered. This is especially true when 
the ambiguity surrounding the question is great These are things that science 
generally ignores due to the need to be sufficiently rigorous. Science is too narrow 
to support organizational decision making. 

KUHN'S PERSPECTIVE: THE SOCIAL STRUCTURE OF SCIENCE 

The concept of the paradigm (Kuhn, 1962, 1977) provides yet another 
construct for exploring the links between institutional research and science. A 
paradigm is a metaphor for understanding the world. Some might even equate it 
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with Weltanschauung (world view). A paradigm is a set of shared beliefs about the 
nature of reality. That set of beliefs determines what is meaningful and important, 
how the important and meaningful things can be examined, and where answers can 
be found. Kuhn refers to those who share such a set of beliefs and share in the 
practice of a scientific specialty a "scientific community" (1962, p. 16). 

Kuhn suggests that a scientific community shares a "characteristic set of 
beliefs and preconceptions" (1962, p. 17). Institutional research has something near 
to a "scientific community" as defined by Kuhn. The goals of encouraging student 
success and maintaining institutional viability are shared by those who fulfill the 
function of institutional research. Many of the techniques used in institutional 
research are shared. The same kinds of problems are the focus of attention of those 
associated with institutional research. And even though findings of studies may not 
be generalizable to other institutions, the processes used to arrive at the results may 
be generalized in that they become the accepted methods of analyzing certain 
information to inform decisions. 

Kuhn also states that members of a scientific community share common 
educational experience and are socialized to share commitments to certain values. 
Even though institutional researchers come from diverse educational backgrounds, 
they usually share some training in the use of statistics, in one or more of the social 
sciences, and in computer usage. They are socialized to the specialization via 
professional organizations, conferences, and professional journals. 

Institutional research is in a pre-paradigm stage; no body of scientific theory 
controls the kinds of questions that can be asked nor the kinds of answers sought. 
This is nothing for institutional researchers to be embarrassed about. According to 
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Kuhn, "it remains an open question what parts of social science have yet 
acquired... [universally accepted] paradigms at all" (Kuhn, 1962, p. 15). Yet 
institutional research does have principles that provide guidance in what is 
appropriate to study. The principlesjnvolve perpetuation of higher education in 
particular contexts. Even though institutional researchers share no single paradigm, 
institutional research has undergone changes on the order of paradigm shifts. 

Each time a new issue emerges as one of the goals of the institution, 
institutional research must adjust to incorporate the concept into investigation of 
each question posed. Diversity is a prime example. Over time, we have seen a 
change in the rhetoric used to describe attempts to reduce racial discrimination and 
increase racial harmony. First, the term was "desegregation." Later it was called 
"integration." Now it is called diversity. Each change has represented a different 
focus for institutional researchers. Each change has resulted in a different treatment 
of information. Since diversity has become a major goal of institutions, all choices 
must be weighed against the possibility of jeopardizing progress toward that goal. 

CONCLUSION 

Because institutional research is dependent upon the politics of higher 
education and upon the social goals of government and communities, identification 
of what is relevant will change. This dynamic nature of institutional research 
demands that the observer shift his or her vision in pattern identification frequently. 
For that reason, institutional researchers are constrained in their attempts to develop 
a cohesive body of knowledge. If institutional research were not so dependent 
upon the perceptions of those wi'hin a particular institutional context, it might be 
possible for a discipline to emerge. 
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However, institutional research is a support function for institutions of 
higher education. An issue is important to a particular institution only when it is 
recognized as an issue. For the most part, other?; set the agenda for institutional 
research. Sometimes it is the government; sometimes it is the school's 
administration. There is little incentive for this to change. 

Institutional research has many hnks with science. It makes use of scientific 
method. It shares elements of science's logical structure. And it is practiced by a 
particular community that shares some beliefs about what is important to study, and 
how to study ■ ..at is important. Nonetheless, institutional research is different 
from science in meaningful ways, the most important of which is the difference in 
purpose. Institutional research uses scientific conventions as tools to achieve its 
purpose of answering "which" questions. That is not to say that institutional 
researchers never participate in science; that is not the point. The "theory" of the 
single institution determines the nature of a study in institutional research. The 
theory of institutions of higher education, that each institution has a mission, goals, 
strategies, and a set of assumptions about how questions should be approached 
(whether or not these have been made explicit), is what creates a community of 
institutional researchers. 

This discussion has only begun the process of identifying ideal types for 
institutional research. Refinements of the structure discussed herein, the 
identification of assumptions, and the recognition of unifying principles in research 
will contribute to greater understanding of institutional research as a unique 
disciplined pursuit and assist in the strengthening of the institutional research 
community. 
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